The present invention relates generally to Internet/Intranet network systems, and in particular, to web caching where Web objects are predictively pre-cached based on time-based pattern of usage.
Web caching is generally well known to those skilled in the art. Generally, a web cache sits between web servers (or remote servers) and a client and watches requests for HTML pages, images and files (known as objects) as they come by, saving a copy for itself. Typically, the web cache resides in a proxy server that sits in a local server. Then if there is another request for the same object, it will use the copy that it has, instead of asking the remote server for it again.
There are two main reasons for using web caches. One is to reduce latency. Because the request is satisfied from the cache (which is closer to the client) instead of the remote server, it takes less time for the client to get the object and display it. This makes web sites seem more responsive. The second reason is to reduce traffic. Because each object is only gotten from the server once, it reduces the amount of bandwidth used by a client. This saves money if the client is paying based on traffic, and keeps their bandwidth requirements lower and more manageable.
Generally, there are two types of web caches: browser caches and proxy caches. The browser cache works by setting a section of a computer""s hard disk to store objects. The browser cache works according to fairly simple rules. It will check to make sure that the objects are fresh, usually once a session. Proxy cache, while it works on a similar principle, is usually located in a local server, a gateway, or a router and can be shared by a number of clients.
All caches have a set of rules that they use to determine when to serve an object from the cache if it""s available. Some of these rules are set by the HTTP protocols and, some are set by the administrator of the cache. (For a full discussion of above, see Caching Tutorial for web Authors and webmasters by Mark Nottingham).
A number of cache replacement strategies are currently suggested as an attempt to optimally reduce latency and/or traffic. For instance, U.S. Pat. No. 6,272,598 describes a cache system that includes a storage that is partitioned into a plurality of storage areas. Each storage area stores one kind of object received from remote sites. The cache system further includes a cache manager which causes objects to be stored in the corresponding storage areas of the storage. The cache manager causes cached objects in each of the storage areas to be replaced in accordance with one of a plurality of replacement policies, each being optimized for one kind of objects.
In another example, the U.S. Pat. No. 6,128,701 describes a system for automatically refreshing documents in a cache, so that each particular document is refreshed no more often and no less often than needed. For each document, the cache estimates a probability distribution of times for client requests for that document and a probability distribution of times for server changes to that document. Times for refresh are selected for each particular document in response to both the estimated probability distribution of times for client requests and the estimated probability distribution of times for server changes.
Yet in another example, the U.S. Pat. No. 5,802,292 describes a method for predictive pre-fetching of objects over a computer network which includes the steps of providing a client computer system, providing a server computer system, a network link to the client computer system, requesting from the server computer system by the client computer system a retrieval of a plurality of objects, retrieving the plurality of objects by the server system, storing the retrieval and an identity of the client computer system in the memory of the server computer system, sending the plurality of objects from the server computer system to the client computer system over the network link, predicting in the server computer system a subsequent retrieval request from the client computer system according to a predetermined criteria, sending the prediction to the client computer system, and pre-fetching by the client computer system an object based on the prediction and other information.
In general, however, a conventional caching method being employed is one where a table keeps track of the requested web page URL (Universal Resource Locator), the time it was last used, and the time stamp of the page. If the table can hold one hundred entries, for instance, then the last hundred web pages visited by the server client using this cache will still be cached locally. If a page is not found in the local cache, the hundredth page is removed from the table, and the URL of the new page is added to the top of the table. If a page is found in the local cache, the cache requests a new page from the remote server, checks the time stamp, and if it is the same as the time stamp of the web page found in the table, stops the transfer and delivers the web page already cached. If the time stamp is different, however, the page is reloaded into the cache and is delivered to the client.
In this prior art method the local server only keeps track of the time stamp of the web page. A web page, however, is typically composed of many elements or objects, such as text files, graphic files, audio files, and video files. Each of these elements or objects has a unique URL and a time stamp, and can be downloaded independently of each other. However, the conventional caching method does not keep track of these individual elements. Consequently, a decision to use a cached web page is an xe2x80x9call or nothingxe2x80x9d deal. Even if only a component, e.g. a graphic file, has changed on a web page, the entire web page is requested from a remote server. Consequently, all of the elements that make up the web page need to be sent again. A significant reduction in latency and/or traffic may be achieved by keeping track of the time stamp of the individual elements of a web page as well as the time stamp of the web page itself, and only requesting those elements which have undergone a change. However, this type of replacement strategy is not available in the current cache systems.
Moreover, in the current caching system, the web pages are cached at the time they are requested. While this type of caching method does help to reduce the overall traffic and latency, it does not take into account the fact that certain web pages are downloaded more heavily during certain times of the day. The latency may further be reduced if the web pages which are popular during a certain period is pre-loaded some time before the xe2x80x9chigh-access periodxe2x80x9d begins. By pre-loading the web pages, the clients will experience fewer delays in accessing the web pages. If the high access period for a particular site happens to be during a heavy congestion period, the web pages may be pre-loaded or pre-fetched into the cache during a less congested period. This will reduce both latency as well as traffic. However, this type of time-based method of predictive pre-fetching of web pages is currently not available.
The present method for predictive caching operation determines a time-based pattern of a high-access period for a web page, and pre-fetches the web page into a cache before the high access period begins. A table is generated where the table comprises a URL, a time of last access and a time stamp of the pre-fetched web page. When a request for a web page is made, the requested web page""s URL and time stamp are checked with the URL and time stamp listed in the table to determine whether any modification has been made. The web page which has been modified is downloaded into the cache.
In another embodiment, the method for predictive caching operation comprises determining a time-based pattern of high-access period for a web page; pre-fetching the web page into a cache before the high access period begins; generating a table, said table comprising a URL, a time of last access and a time stamp of the pre-fetched web page, said table further comprising a URL, time of last access and time stamp of elements found on the pre-fetched web page; when a request for a web page is made, checking the requested web page""s URL and time stamp and URL and time stamp of the requested web page""s corresponding elements with the URL and time stamp listed in the table to determine whether any modification has been made; and selectively downloading in a cache only those elements which are deemed to have been modified.